Attorney Docket 2736-122 



WHAT IS CLAIMED IS: 

1 . A method for administering a constructed response assessment 
item to a respondent: 

defining a stimulus response stem requiring the respondent to 
demonstrate knowledge of a relationship between two or more concepts; 

providing the respondent at least one device uniquely associated with 
each of the two or more concepts, at least one device having at least one 
attribute that can be selectively manipulated by the respondent to change a 
relationship of that device with respect to at least one other device to construct 
an arrangement of devices representing a relationship between the concepts 
with which the devices in the arrangement are associated; 

defining a scoring rubric for the constructed response assessment item 
including a conceptual pattern corresponding to a relationship between the two 
or more concepts; 

observing the selectively manipulable attributes of the devices in the 
arrangement constructed by the respondent; 

converting the observed attributes into a conceptual pattern associated 
with the arrangement constructed by the respondent; and 

comparing the conceptual pattern of the arrangement constructed by 
the respondent with the conceptual pattem of the scoring rubric to determine 
whether or not the conceptual pattem associated with the arrangement 
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constructed by the respondent matches the conceptual pattern of the scoring 
rubric. 

2. The method of claim 1, wherein said rubric includes a 
conceptual pattern corresponding to a correct relationship between the two or 
more concepts. 

3. The method of claim 1, wherein the devices comprise physical 
items that the respondent can physically manipulate and/or position with 
respect to each other to create a spatial arrangement of the devices 
demonstrating a relationship between the concepts with which the devices are 
associated. 

4. The method of claim 1, wherein the devices comprise virtual 
items displayed on a computer that the respondent can electronically 
manipulate and/or position with respect to each other to create a virtual 
arrangement of the devices demonstrating a relationship between the concepts 
with which the devices are associated. 

5. The method of claim 1 , wherein the attribute of each device 
that can be manipulated includes one or more of size, length, height, spatial, 
orientation, color, direction of movement, speed of movement, sound, sound 
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volume, texture, pattern, smell, taste, elasticity, weight, transparency, 
absorbency, reflectivity, shape, electrical charge, magnetism, temperature, 
conductivity, composition, intensity, perspective, emotion, and time. 

6. The method of claim 1, further comprising defining at least one 
implication corresponding to each pattern of the rubric, wherein each 
implication comprises information relating to an outcome that occurs as the 
result of a pattern being found in the student response to an item. 

7. A method of creating a constructed response assessment item: 
defining a stimulus response stem requiring a respondent to 

demonstrate knowledge of a relationship between two or more concepts; 

defining at least one device uniquely associated with each of the two or 
more concepts, at least one device having at least one attribute that can be 
selectively manipulated by the respondent to change a relationship of that 
device with respect to at least one other device to construct an arrangement of 
devices representing a relationship between the concepts with which the 
devices in the arrangement are associated; and 

defining a scoring rubric for the constructed response including a 
conceptual pattern corresponding to a relationship between the two or more. 
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8. The method of claim 7, wherein said rubric includes a 
conceptual pattern corresponding to a correct relationship between the two or 
more concepts. 

9. The method of claim 7, wherein the devices comprise physical 
items that the respondent can physically manipulate and/or position with 
respect to each other to create a spatial arrangement of the devices 
demonstrating a relationship between the concepts with which the devices are 
associated. 

10. The method of claim 7, wherein the devices comprise virtual 
items displayed on a computer that the respondent can electronically 
manipulate and/or position with respect to each other to create a virtual 
arrangement of the devices demonstrating a relationship between the concepts 
with which the devices are associated. 

1 1 . The method of claim 7, wherein the attribute of each device 
that can be manipulated includes one or more of size, length, height, spatial, 
orientation, color, direction of movement, speed of movement, sound, sound 
volume, texture, pattern, smell, taste, elasticity, weight, transparency, 
absorbency, reflectivity, shape, electrical charge, magnetism, temperature, 
conductivity, composition, intensity, perspective, emotion, and time. 
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12. The method of claim 7, further comprising defining at least one 
implication corresponding to each pattern of the rubric, wherein each 
implication comprises information relating to an outcome that occurs as the 
result of a pattern being found in the student response to an item. 

13. The method of claim 7, wherein defining at least one device 
comprises: 

selecting a template defining a plurality of devices and attributes of 
each device that can be manipulated; and 

selecting one of said devices to be uniquely associated with each of 
said two or more concepts. 

14. A system for administering a constructed response assessment 
item to a respondent, said system comprising: 

a content creation engine for defining a stimulus response stem 
requiring the respondent to demonstrate knowledge of a relationship between 
two or more concepts, said content creation engine comprising: 

a concept creator constructed and arranged to allow an item 
designer to defined the two or more concepts; 

a template selector constructed and arranged to enable the item 
designer to select a template defining devices with which each of the 
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two or more concepts will be associated and attributes of the devices 
that can be manipulated by the respondent; and 

a representation linker constructed and arranged to enable the 
item designer to uniquely associate at least one of the devices defined 
by said template selector with each of the two or more concepts; 
an administration engine for administering the constructed response 
item to the respondent, said administration engine comprising: 

a representation engine constructed and arranged to present 
devices selected to represent the two or more concepts to a respondent; 

a manipulation engine constructed and arranged to enable the 
respondent to manipulate attributes of the devices; and 

an attribute acquisition engine constructed and arranged to 
retrieve and record attributes of the manipulated devices; 
a rubric engine for creating a rubric for evaluating a response by the 
respondent, said rubric engine comprising: 

a pattem defmer constructed and arranged to enable the item 
designer to define one or more rubric pattems defining device 
arrangements that correspond to correct and incorrect responses that 
are possible with the templates selected using said template selector; 
and 
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an implication selector constructed and arranged to enable the 
designer to associate one or more implications with each of the 
patterns defined by said pattern definer; and 

a scoring engine for scoring a response created by the respondent, said 
scoring engine comprising: 

a pattern recognition and comparison engine constructed and 
arranged to convert the attributes of the manipulated devices recorded 
by said attribute acquisition engine and to convert the attributes to a 
response pattem and to compare the response pattern to the one or 
more rubric patterns defined by said pattem definer to identify any 
match that may exist between the response pattem and the one or more 
rubric patterns; 

an implication selector constructed and arranged to select at 
least one of the one or more implications associated with any rubric 
patterns which match the response pattem; and 

a results engine constructed and arranged to return desired 
response results based on the implication selected by the implication 
selector. 

15. A system for creating a constructed response assessment item 
comprising: 
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a content creation engine for defining a stimulus response stem 
requiring the respondent to demonstrate knowledge of a relationship between 
two or more concepts, said content creation engine comprising: 

a concept creator constructed and arranged to allow an item 
designer to defined the two or more concepts; 

a template selector constructed and arranged to enable the item 
designer to select a template defining devices with which each of the 
two or more concepts will be associated and attributes of the devices 
that can be manipulated by the respondent; and 

a representation linker constructed and arranged to enable the 
item designer to uniquely associate at least one of the devices defined 
by said template selector with each of the two or more concepts; and 
a rubric engine for creating a rubric for evaluating a response by the 
respondent, said rubric engine comprising: 

a pattem definer constructed and arranged to enable the item 
designer to define one or more rubric patterns defining device 
arrangements that correspond to correct and incorrect responses that 
are possible with the templates selected using said template selector; 
and 

an implication selector constructed and arranged to enable the 
designer to associate one or more implications with each of the 
patterns defined by said pattem definer. 
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16. A system for administering a constructed response assessment 
item to a respondent, said system comprising: 

content creation means for defining a stimulus response stem requiring 
the respondent to demonstrate knowledge of a relationship between two or 
more concepts; 

item administration means for administering the constructed response 
item to the respondent; 

rubric means for creating a rubric for evaluating a response by the 
respondent; and 

scoring means for scoring a response created by the respondent. 

17. A system for administering a constructed response assessment 
item requiring a respondent to demonstrate knowledge of a relationship 
between two or more concepts by manipulating attributes of devices uniquely 
associated with each of said two or more concepts, said system comprising: 

an administration engine for administering the constructed response 
item to the respondent, said administration engine comprising: 

a representation engine constructed and arranged to present 
devices selected to be uniquely associated with each of the two or more 
concepts to a respondent; 

a manipulation engine constructed and arranged to enable the 
respondent to manipulate attributes of the devices; and 
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an attribute acquisition engine constructed and arranged to 
retrieve and record attributes of the manipulated devices; and 
a scoring engine for scoring a response created by the respondent, said 
scoring engine comprising: 

a pattern recognition and comparison engine constructed and 
arranged to convert the attributes of the manipulated devices recorded 
by said attribute acquisition engine and to convert the attributes to a 
response pattern and to compare the response pattern to one or more 
predefined rubric patterns which define device arrangements that 
correspond to correct and incorrect responses to identify any match 
that may exist between the response pattern and the one or more rubric 
patterns. 
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